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1.0  EXECUTIVE  SUMMARY 


The  objective  of  this  project  was  to  demonstrate  and  certify  a  fixed-wing  platform  for  low-cost, 
high-resolution  wide  area  assessment  (WAA)  of  former  and  active  military  facilities 
contaminated  with  unexploded  ordnance  (UXO).  The  Minimum  Altitude  Remote  Sensing 
(MARS)  airborne  UXO  mapping  system  was  developed,  successfully  tested,  and  deployed  in 
Europe  by  SeaTerra  GmbH  in  Germany.  The  principal  objectives  of  this  demonstration  were  to 
test  and  evaluate  the  MARS  system  in  the  United  States  and  compare  the  performance,  results, 
and  cost  to  HeliMag  technology.  The  site  selected  for  demonstration  was  the  Former  Kirtland 
Precision  Bombing  Range  (KPBR),  located  near  Albuquerque,  New  Mexico. 

Sky  Research,  Inc.  (SKY)  acquired  the  CT  Short  Wing  (SW)  model  light  sport  aircraft  (LSA)  for 
this  demonstration.  This  model  lightweight  aircraft,  a  slightly  different  model  than  the  one  used 
by  SeaTerra  in  Europe,  did  not  affect  the  deployment  of  the  technology.  The  aircraft  has  an 
8.9  m  wingspan,  a  length  of  6.2  m,  and  can  fly  at  a  minimum  height  above  ground  of  2  m.  It  uses 
modem  German  glider  structural  design  techniques  and  is  essentially  all-composite  glass  fiber 
reinforced  plastic  with  very  little  ferrous  metal  and  a  low  signature  footprint.  An  array  of  six 
Geometries  Model  822A  cesium  vapor  magnetometer  sensors  was  mounted  on  a  sensor  boom 
attached  to  the  aircraft  wings.  The  sensors  were  deployed  at  separations  between  1.15  and  2.0  m. 
SeaTerra ’s  AGS  MK3  frequency  counter  and  data  acquisition  system  (DAS)  were  used  to  record 
data  at  between  140  to  160  Hertz  (Hz)  with  a  ~0.2  nanotesla  (nT)  noise  floor  on  the  outer  sensors 
and  0.5-0. 7  nT  noise  floor  on  the  inner  sensors  (after  processing). 

Daily  flight  durations  were  planned  to  be  approximately  6  hours  per  day  (assuming  an  8-hour 
daily  window  for  flying).  Because  of  unfavorable  weather  conditions  (frequent  high  winds  and 
rain  or  thunderstorms),  density  altitude1,  and  thennal  effects  that  were  encountered,  it  was  not 
possible  to  safely  operate  the  CT  as  many  hours  per  day  as  planned.  Daily  flight  durations  ranged 
from  1.4  hours  to  a  maximum  of  5.2  hours,  with  an  average  of  3.5  hours  for  days  when 
conditions  for  flying  were  favorable.  There  were  3  days  that  no  flights  were  conducted  due  to 
rain  and/or  high  winds.  During  the  survey,  data  were  collected  over  2,856  acres. 

A  number  of  105-mm  high-explosive  anti-tank  (HEAT)  105-mm  and  155-mm  projectiles  were 
blind-seeded  in  an  area  south  of  the  Double  Eagle  Airport.  For  manually  selected  target  picks, 
there  were  2.1  anomalies  per  acre,  and  detection  probabilities  were  11,  60,  and  93%  for  the  105- 
mm  HEAT,  105-mm  projectile,  and  155-mm  projectile,  respectively.  Using  an  automatic 
detection  algorithm  and  a  threshold  of  5  nT,  anticipated  detection  probabilities  were  62,  7 1  and 
100%  for  the  above  three  items  with  19  alarms  per  acre. 

A  HeliMag  survey  was  flown  over  the  survey  area  in  2005,  and  portions  of  the  site  were  also 
covered  by  a  towed-array  survey.  When  target  picking  using  the  total-gradient  data,  the  MARS 
system  experienced  an  inflection  point  at  around  4  nT/m  (where  the  number  of  alanns  increases 
markedly),  compared  to  3  nT/m  for  the  HeliMag  data. 


1  This  is  the  altitude  in  the  International  Standard  Atmosphere  at  which  the  air  density  would  be  equal  to  the  actual 
air  density  at  the  place  of  observation. 


We  compared  the  MARS  and  HeliMag  anomaly  density  maps  in  two  regions  of  the  northern  part 
of  the  site  (Areas  1  and  2).  In  Area  1,  the  MARS  and  HeliMag  derived  densities  were 
comparable,  and  both  predicted  a  target  of  approximately  the  same  size,  shape,  and 
concentration.  In  Area  2,  the  density  estimates  were  considerably  different  with  the  MARS 
densities — on  average,  3.5  times  lower  than  the  HeliMag.  This  difference  was  largely  attributed 
to  the  approximately  1.0  m  higher  elevation  of  the  MARS  sensors  compared  to  the  HeliMag.  The 
differences  were  particularly  marked  around  a  fence  that  cut  across  a  high  concentration  target 
region.  The  CT  aircraft  had  to  fly  higher  over  the  obstacle  and  took  longer  to  return  to  its  normal 
survey  altitude  than  the  helicopter. 

Using  the  towed-array  data  as  groundtruth,  the  HeliMag  system  exhibited  detection  probabilities 
(Pd)  of  Pd=0.9  on  Areas  1  and  2  at  five  false-alarms  per  acre.  In  contrast,  for  Area  1  the  MARS 
Pd=0.7  at  live  alanns  per  acre  and  rose  slowly  to  Pd=0.85  at  30  alarms  per  acre.  MARS 
detection  performance  was  significantly  worse  for  Area  2,  with  Pd=0.6  at  around  five  false 
alarms  per  acre,  rising  to  Pd=0.8  at  20  false  alarms  per  acre. 

The  pilots  were  able  to  consistently  fly  the  aircraft  at  the  target  2  to  3  m  above  ground  level. 
However,  the  poor  detection  perfonnance  in  Area  2,  especially  in  the  area  where  the  pilot  had  to 
avoid  a  fence,  indicates  that  surface  tracking  is  more  problematic  for  the  CT  than  a  helicopter. 

The  main  difference  in  cost  between  MARS  and  HeliMag  is  in  the  cost  of  the  aircraft,  with  daily 
helicopter  costs  approximately  five  times  that  of  MARS.  Depending  on  the  distance  required  for 
mobilization  and  the  size  of  the  area  surveyed,  a  MARS  survey  costs  5 1  to  66%  of  a  HeliMag 
survey.  For  large  sites  (10,000  acres)  with  small  mobilization  distances  (<  4  hours),  MARS 
would  cost  about  $55  per  acre.  For  smaller  sites  (1,000  acres)  and  greater  mobilization  distances 
(>  16  hours),  MARS  would  cost  approximately  $134  per  acre. 
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2.0 


INTRODUCTION 


2.1  BACKGROUND 

This  project  was  conducted  to  demonstrate  and  certify  a  fixed-wing  platform  for  deploying  low 
altitude  remote  sensing  technologies  that  can  be  used  to  help  meet  the  increasing  demand  on  the 
Department  of  Defense  (DoD)  for  low-cost,  high-resolution  wide  area  assessment  (WAA)  of 
former  and  active  military  facilities  contaminated  with  unexploded  ordnance  (UXO).  The 
objective  of  the  demonstration  of  the  Minimum  Altitude  Remote  Sensing  (MARS)  system  is  to 
evaluate  the  ability  of  this  platfonn  and  technology  to  characterize  large  sites  cost  efficiently, 
reliably,  and  safely.  The  MARS  airborne  UXO  mapping  system  was  developed,  successfully 
tested,  and  deployed  in  Europe  by  SeaTerra  GmbH  in  Germany.  Sky  Research,  Inc.  (SKY)  is 
demonstrating  and  certifying  this  system  (Figure  1)  for  application  in  the  United  States  in 
partnership  with  SeaTerra. 

The  site  selected  for  demonstration  is  the  Former  Kirtland  Precision  Bombing  Range  (KPBR) 
located  near  Albuquerque,  New  Mexico,  in  order  to  provide  the  ability  to  compare  performance, 
results,  and  cost  with  another  low  altitude  WAA  technology,  helicopter  magnetometry 
(HeliMag).  HeliMag  was  previously  demonstrated  at  the  site  as  part  of  the  Environmental 
Security  Technology  Certification  Program  (ESTCP)  WAA  Pilot  Program.  In  addition,  the 
results  of  this  demonstration  will  be  integrated  into  the  SKY  WAA  Geographic  Information 
System  (GIS)  with  the  WAA  data  collected  at  the  site  (HeliMag,  Fight  Detection  and  Ranging 
[FiDAR],  orthophotography,  and  ground-based  digital  geophysical  mapping  [DGM]  data)  to 
facilitate  data  analysis,  historical  information  integration,  and  data  access. 


Figure  1.  The  CT  SW  Light  Sport  Aircraft  Collecting  Data  at  the  Former  Kirtland 

Precision  Bombing  Range. 

2.2  OBJECTIVES  OF  THE  DEMONSTRATION 

The  principal  objectives  of  the  MARS  project  are  to  test  and  evaluate  the  MARS  system  in  the 
U.S.  and  compare  the  performance,  results  and  cost  to  HeliMag  technology.  Similar  to  HeliMag, 
MARS  is  expected  to  demonstrate  efficient,  low-altitude  DGM  capabilities  for  metal  detection  at 
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a  resolution  approaching  that  of  ground  survey  methods,  limited  primarily  by  terrain,  vegetation, 
and  topographic  inhibitions  to  safe  low  altitude  flight. 

2.3  REGULATORY  DRIVERS 


The  Former  KPBR  is  a  WWII-era  former  military  training  facility  located  approximately  10 
miles  west  of  Albuquerque.  The  site  is  classified  as  a  Formerly  Used  Defense  Site  (FUDS). 
ESTCP  established  a  6,500  acre  demonstration  plan  sub-area  for  the  WAA  Pilot  Program  in 
2005,  and  the  MARS  demonstration  survey  also  was  conducted  within  the  WAA  boundary. 
Results  from  the  data  analysis  for  the  WAA  Pilot  Program  confirmed  the  presence  of  three 
precision  bombing  targets  (N2,  N3,  and  New  Demolitions  Impact  Area  [NDIA])  and  a  simulated 
oil  refinery  target  (SORT),  and  several  additional  areas  of  interest.  Currently,  most  of  the  study 
area  is  undeveloped;  however,  portions  of  the  area  are  planned  for  commercial  or  industrial 
development  within  the  next  decade.  The  KPBR  site  also  encompasses  the  Double  Eagle  Airport, 
which  is  owned  by  the  City  of  Albuquerque.  The  airport  is  very  active  and  there  are  plans  for 
future  expansion  of  aviation  facilities  and  infrastructure,  as  well  as  promotion  of  business 
development  to  attract  aviation-related  businesses.  It  will  be  necessary  to  ensure  that  any 
munitions-related  contamination  is  fully  characterized  and  remediated  to  allow  for  airport 
expansion  and  business  development. 
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3.0  TECHNOLOGY 


3.1  TECHNOLOGY  DESCRIPTION 

The  technology  components,  including  the  aircraft,  and  all  sensor  and  positioning  components 
and  associated  electronics  are  described  in  the  following  sections. 

3.1.1  Airborne  Platform 

SKY  acquired  the  CT  Short  Wing  (SW)  model  light  sport  aircraft  (LSA)  for  this  demonstration. 
This  model  lightweight  aircraft,  a  slightly  different  model  than  the  one  used  by  SeaTerra  in 
Europe,  did  not  affect  the  deployment  of  the  technology.  The  aircraft  has  an  8.9  m  wingspan,  a 
length  of  6.2  m,  and  can  fly  at  a  minimum  height  above  ground  of  2  m.  It  uses  modern  German 
glider  structural  design  techniques  and  is  essentially  all-composite  glass  fiber  reinforced  plastic 
with  very  little  ferrous  metal  and  a  low  signature  footprint.  The  aircraft  serves  as  the  platfonn  for 
deployment  of  the  sensor  and  positioning  technologies  used  for  this  demonstration  (Table  1). 

Table  1.  MARS  Technology  Components 


Technology  Component 

Specifications 

Geophysical  sensors 

6  Geometries  G-822A  cesium  vapor  magnetometers 

Global  Positioning  System  (GPS)  equipment 

Trimble  AgGPS  model  132  for  navigation  and  Trimble 
MS750  system  with  2  GPS  receivers  for  precise  real-time 
kinematic  (RTK)  positioning 

Altimeters 

1  Reigel  laser  altimeter 

Inertial  measurement 

SeaTerra  compensation  system 

Magnetic  compensation  (and  redundant  attitude 
information) 

SeaTerra  digital  compass  and  proprietary  magnetic 
compensation  system 

Data  acquisition  system  (DAS)  and  frequency 
counter 

AGS  MK3  (SeaTerra  proprietary  DAS  and  frequency 
counter):  Better  than  0.1  nanoTesla  (nT)  resolution  at,  150 
Hz:  0.1  to  0.7  nT  noise  floor 

Aircraft 

Flight  Designs  CT  SW 

3.1.2  Airborne  Sensors  and  Boom 

An  array  of  six  Geometries  Model  822A  cesium  vapor  magnetometer  sensors  are  mounted  on  the 
sensor  boom  (Figure  2).  The  sensor  system  is  mounted  on  the  CT  SW  with  support  structures 
fixed  to  the  aircraft  wings.  We  attempted  to  maintain  consistent  1.4-m  spacing  between  sensors 
but  found  that  we  could  achieve  much  lower  noise  levels  if  we  moved  several  of  the  sensors.  In 
particular,  the  two  inner  sensors  (numbers  3  and  4)  were  moved  so  they  were  2.0  m  apart,  with 
the  other  sensors  between  1.15  and  1.4  m  apart.  Sensor  2  was  moved  a  little  closer  to  sensor  1  to 
place  it  further  from  the  laser  altimeter  which  appeared  to  be  causing  a  small  increase  in  the 
noise  level  of  that  sensor. 

SeaTerra ’s  AGS  MK3  frequency  counter  and  DAS  were  used  to  record  data  at  around  160  Hz 
with  a  ~0.2  nT  noise  floor2  after  processing. 


2  The  inner  sensors  3  and  4  exhibited  noise  levels  between  0.45  and  0.7  nT. 
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This  system  was  thoroughly  tested  on  the  MARS  platform  during  field  trials  and  production 
work  within  Germany.  As  part  of  this  project,  the  aircraft  modifications  were  required  to  be 
approved  for  deployment  in  the  United  States.  Consultation  with  the  Federal  Aviation 
Administration  (FAA)  resulted  in  direction  to  obtain  the  manufacturer’s  approval  of  the 
modifications.  Because  the  CT  falls  in  the  LSA  category,  the  FAA  does  not  have  any 
requirements  or  authority  to  approve  modifications.  SKY  provided  the  specifications  and 
engineering  drawings  for  the  sensor  boom  and  other  components  to  the  Ukrainian  manufacturer 
for  their  evaluation  and  testing  to  obtain  the  required  approvals. 

3.1.3  Positioning  Technologies 

For  the  demonstration  survey,  we  also  deployed  two  Trimble  MS750  GPS  units  configured  in  a 
Moving  Base-Rover  mode  to  provide  National  Marine  Electronics  Association  GGK  (NMEA 
time,  position,  position  type,  and  dilution  of  precision  [DOP]  designation)  positions  at  20  Hz 
GPS  and  Advanced  Visual  RISC  [Reduced  Instruction  Set  Computer]  (AVR)  orientation  data  at 
10  Hz  (this  is  the  same  system  used  successfully  for  the  HeliMag  technology).  The  system 
requires  two  MS750  Marine  GPS  receivers,  two  compact  L1/L2  GPS  antennas,  12-24  Volt 
power,  and  RTK  corrections.  The  GPS  antennas  were  flush-mounted  close  to  the  wing  tips  on 
both  sides  of  the  aircraft,  in  line  with  the  magnetometer  boom.  This  provided  a  baseline  of  6  m. 
The  NMEA  GPS  data  provide  the  latitude  and  longitude  of  antennas  while  the  AVR  data  return 
the  platfonn  roll  and  yaw  (an  equivalent  concept  is  used  very  successfully  in  the  HeliMag 
system). 

A  Reigal  laser  altimeter  recorded  the  aircraft  height  above  ground  at  cm-level  accuracy  at  a 
sample  rate  of  50  Hz.  Since  the  fixed-wing  system  does  not  change  altitude  quickly,  the  altitude 
values  show  nominal  variation,  and  the  normal  flight  height  above  ground  is  2-3  m.  The  3-D 
position  of  each  data  point  was  utilized  in  subsequent  processing  (e.g.,  the  magnetic  inversion 
modules  in  UXOLab  allow  inversion  of  magnetic  data  with  variable  altitudes). 

3.1.4  Development  and  Prior  Testing  of  the  Technology 

SeaTerra  GmbH,  partnered  with  SKY  for  this  project,  developed  and  has  deployed  the  MARS 
system.  The  system  was  used  to  survey  nearly  8,000  survey  acres  at  sites  in  Europe  prior  to 
initiating  this  project.  Adaptation  of  the  sensor  boom  for  the  CT  SW  and  certification  of  the 
system  for  use  in  the  United  States  were  conducted  from  May  to  November,  2006.  Pre¬ 
demonstration  test  flights  were  conducted  in  December,  2006  and  April,  2007  to  confirm  that  the 
system  was  operating  within  specifications. 
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Figure  2.  Schematic  of  the  MARS  Sensor  Layout. 

(In  this  diagram  the  sensors  are  shown  1.25  m  apart.  For  this  demonstration, 
we  used  a  sensor  spacing  of  between  1.1  and  2.0  m.) 

The  MARS  system  was  tested  by  the  German  government  on  military  ranges  in  southern 
Germany.  In  addition,  it  was  used  to  conduct  four  commercial  WAA  UXO  surveys  covering 
nearly  8,000  acres  in  Germany  between  2001  and  2006.  A  summary  of  the  previous  application 
of  the  technology  to  munitions  and  explosives  of  concern  (MEC)  site  characterization  is 
provided  in  Table  2.  Achieved  productivity  on  these  sites  ranged  from  250-500  acres/day.  The 
system  proved  to  be  very  accurate,  efficient,  reliable,  and  suitable  even  in  areas  that  were 
considered  non-ideal.  The  statistical  approach  of  calculating  relative  density  distribution 
coefficients  has  proven  to  be  reliable,  and  the  results  correlate  with  ground  truth  surveys  and 
excavations.  Since  an  array  of  cesium  vapor  magnetometers  are  the  main  detection  sensors,  the 
ability  to  detect  small  ferrous  objects  is  diminished  as  the  separation  between  the  sensor  and  the 
target  increases.  However,  at  2  m  elevation,  MARS  is  capable  of  detecting  2.75-inch  rockets  and 
larger  items  of  interest. 


Table  2.  MARS  Survey  Sites. 


German 

Training 

Range 

Total 

Area 

(acres) 

Survey 

Area 

(acres) 

Vegetation 

Condition 

— 

Facility 

Use 

Munsingen 

Government 

16,055 

3,952 

40%  wooded 

250  m 

150  years:  Artillery, 
infantry,tank  training, 
bombing 

Eggersdorf 

Private 

investor 

6,175 

988 

Free 

0  m 

100  years:  Military  airfield 

Oranienburg 

Government 

2,470 

494 

50%  wooded 

2  m 

100  years:  Air  Force  Base 

Trampe 

Germany  - 
EPA 

14,820 

1,976 

30%  wooded 

30  m 

50  years:  Tank  training 

Development  of  the  system  was  conducted  under  this  project  to  fabricate  and  certify  a  sensor 
boom  that  could  be  mounted  to  the  CT  SW.  Because  of  the  structural  differences  between  the 
SW  and  Gennan-based  CT,  an  engineering  analysis  was  required  to  select  the  appropriate 
mounting  locations  on  the  wings  and  design  the  sensor  boom  struts.  The  design  was  conducted  in 
conjunction  with  the  Ukrainian  manufacturer,  and  certification  of  the  modified  boom  design  was 
granted  upon  successful  structural  and  weight  tests. 

Table  3  summarizes  the  results  of  a  final  high-altitude  characterization  test  with  the  system 
configured  and  ready  for  the  demonstration.  A  20-second  segment  of  the  flight,  with  the  aircraft 
flying  straight,  was  used  to  characterize  the  sensor  noise  levels.  The  noise  levels  (after 
processing)  in  sensors  3  and  4  were  reduced  by  a  little  less  than  half  of  the  values  found  during 
test  flights  conducted  in  December  2006.  The  values  for  sensors  1,  2,  4,  and  6  were  between  0.14 
and  0.19  nT.  All  values  were  less  than  our  demonstration  objective  of  less  than  1  nT. 

Moving  the  inner  sensors  slightly  further  apart  (as  well  as  replacing  the  ferrous  bolts  on  the 
wheels)  significantly  reduced  the  effects  of  heading  on  the  magnetic  field  recorded  by  these 
sensors  (Table  3).  During  the  execution  of  a  turn  (that  was  very  similar  to  the  turn  analyzed  in 
December  2006),  sensors  3  and  4  showed  less  variation  relative  to  sensor  1  than  the  previous  test 
and  had  a  lower  bias  compared  to  that  previous  test  (potentially  due  to  the  removal  of  the 
pennanent  magnetic  field  caused  by  the  bolts  on  the  wheel). 

Table  3.  Heading  Effects  and  Noise  Levels  from  High  Altitude  Test  Flights 
in  December  2006  and  May  2007. 


Sensor 

Maximum  difference  (nT)  relative  to 
sensor  1  (during  turn) 

Standard  deviation  (nT) 
(straight  line  segment) 

Dec  06 

May  07 

Dec  06 

May  07 

Sensor  1 

0.0 

0.0 

0.10 

0.16 

Sensor  2 

175 

212 

0.23 

0.14 

Sensor  3 

1270 

489 

1.19 

0.67 

Sensor  4 

883 

359 

0.73 

0.45 

Sensor  5 

162 

195 

0.29 

0.19 

Sensor  6 

175 

167 

0.22 

0.13 
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3.2  ADVANTAGES  AND  LIMITATIONS  OF  THE  TECHNOLOGIES 


As  with  all  characterization  technologies,  site  specific  advantages  and  disadvantages  exist  that 
dictate  the  level  of  success  of  their  application.  The  general  advantages  of  MARS  technology 
include: 


•  The  ability  to  characterize  very  large  areas 

•  Lower  cost  as  compared  to  ground-based  and  helicopter-based  DGM  methods. 

Because  of  the  lower  costs  associated  with  the  aircraft  maintenance  and  acquisition  costs  and 
much  lower  fuel  consumption,  the  MARS  system  is  expected  to  demonstrate  a  much  lower  cost 
to  operate  than  HeliMag. 

There  are  advantages  and  disadvantages  in  using  a  fixed-wing  platfonn.  Open  areas  are  very 
suitable  for  the  fixed-wing  systems.  MARS  can  be  flown  quite  flexibly,  but  obstacles  such  as 
trees  and  power  lines  pose  a  safety  hazard  and  have  to  be  flown  over,  producing  data  gaps.  In 
these  conditions  the  helicopter  platfonn,  the  primary  alternative  technology,  will  outperform 
MARS.  The  topography  of  a  survey  site  plays  a  role  as  well.  If  the  area  is  slightly  hilly,  MARS 
has  no  deployment  limitations.  However,  mountainous  areas  and  steep  slopes  are  not  suitable  for 
fixed-wing  systems.  An  advantage  of  MARS  is  the  ability  to  fly  more  constant  flight  paths  and 
straighter  survey  lines.  An  advantage  of  the  fixed-wing  platfonn  is  that  rapid  positioning  changes 
(change  of  direction  and  speed)  do  not  occur  frequently,  and  therefore  problems  in  positioning 
are  generally  avoided. 

For  all  airborne  surveys,  the  largest  single  factor  affecting  the  survey  costs  is  the  cost  of 
operating  the  survey  aircraft  and  sensors  at  the  site.  These  equipment  costs  are  related  to  capital 
value,  maintenance,  overhead,  and  direct  operating  costs  of  the  sensor  and  aircraft  systems. 
Mobilization  to  and  from  the  site  increases  costs  as  distance  traveled  increases,  and  flexibility  of 
scheduling  is  critical  in  detennining  whether  mobilization  and  deployment  costs  can  be  shared 
across  projects.  In  addition,  low  altitude  surveys  are  limited  by  topography  and  vegetation  and 
therefore  airborne  technologies  can  be  deployed  only  to  sites  with  suitable  conditions. 

An  advantage  of  the  MARS  system  is  the  low-noise  aspect  of  this  platform.  The  CT  SW  aircraft 
is  made  entirely  of  fiberglass  composite  materials;  the  only  major  metallic  component  is  the 
engine.  Since  the  rotation  of  the  engine  is  very  high-speed,  it  only  creates  a  high  frequency  noise 
to  the  data,  which  is  above  the  frequency  of  the  signal  from  items  of  interest  in  the  ground.  Most 
of  the  fixed  metallic  components  of  the  system  were  replaced  by  nonmagnetic  components.  The 
remaining  metal  parts  are  the  engine  and  the  rescue  system  of  the  plane,  which  are  both  fixed  and 
create  a  constant  signature  depending  on  the  movement  of  the  plane  in  the  magnetic  field.  This 
heading  dependent  signature  can  be  removed  with  the  use  of  appropriate  high-pass  filters. 
Careful  attention  has  been  given  to  the  relative  timing  of  the  different  sensors  in  the  MARS 
system  to  avoid  time  delays  or  synchronization  problems.  Calibration  loops  flown  in  high 
altitude  provide  estimates  of  plane  noise  in  correlation  with  the  plane  movement  and  orientation. 
Typical  noise  levels  during  operation  are  ~0.3  nT.  This  is  the  noise  in  the  raw  data,  which 
fortunately  has  a  very  high  frequency.  A  frequency  domain  filter  is  applied  to  produce  data  in  the 
0.1  to  0.2  nT  range  .  These  are  comparable  to  noise  levels  achieved  in  the  HeliMag  systems.  For 


3  For  the  inner  sensors  we  could  only  achieve  noise  floors  between  0.5  and  0.7  nT. 
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instance,  our  analysis  of  the  Hughes  500  helicopter  as  part  of  the  ESTCP  MM0535  deployment 
to  Toussaint  River  revealed  noise  levels  of  0.07  to  0.1 1  nT. 

An  additional  consideration  for  the  MARS  system  is  the  window  for  safe  flying  conditions  for  a 
fixed-wing  aircraft.  Several  additional  factors  more  significantly  affect  safe  operation  of  a 
lightweight,  fixed-wing  aircraft  compared  to  a  helicopter.  These  include: 

•  Windspeeds  less  than  approximately  15  mph  (~13  knots)  are  considered  safe  and 
allow  the  pilot  to  maintain  planned  flight  lines  without  major  deviations. 

•  Ground  effects,  caused  by  the  reduction  of  induced  drag  when  an  airplane  is 
flown  at  slow  speed  very  near  the  ground  surface  can  present  a  hazard.  Ground 
effect  exerts  an  influence  only  when  the  airplane  is  flown  at  an  altitude  no  greater 
than  its  wing  span.  The  effect  increases  as  the  aircraft  descends  closer  to  the 
ground,  with  the  most  significant  effects  occurring  at  a  height  of  one-half  the 
wingspan  above  the  ground.  It  can  present  a  hazard  for  low-level  flight  because 
the  varying  drag  must  be  corrected  for  as  the  aircraft  changes  altitude  to  adjust  to 
varying  terrain,  obstacles,  or  for  other  reasons,  and  there  is  little  room  for  error  in 
making  adjustments  at  such  low  altitudes.  The  optimal  survey  altitude  falls  within 
this  range. 

•  Density  altitude  is  defined  as  the  pressure  altitude  corrected  for  nonstandard 
temperature  variations.  A  simple  definition  of  density  altitude  is  “the  altitude  the 
airplane  thinks  it  is  at,  and  performs  in  accordance  with.”  High  density  altitude 
can  significantly  affect  flight  characteristics,  reducing  the  lift  and  power  output  of 
the  aircraft  and  limiting  safe  flight  operations,  particularly  for  low-level  flight. 
Both  altitude  and  temperature  affect  the  density  altitude.  The  KPBR  site  is  at 
around  5,800  feet  in  elevation  (1,770  meters). 

The  ground  effects  and  density  altitude  issues  also  affect  helicopter  operation  but  less 
significantly  than  the  fixed-wing  aircraft.  This  has  the  result  of  more  limited  flight  time  due  to 
site  altitude  and  weather  conditions  and  has  the  potential  to  offset  the  advantage  of  lower 
operational  costs  to  some  degree. 
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4.0  PERFORMANCE  OBJECTIVES 


Performance  objectives  are  a  critical  component  of  this  demonstration  because  they  provide  the 
basis  for  evaluating  the  perfonnance  and  costs  of  the  technology.  For  the  MARS  project,  both 
primary  and  secondary  perfonnance  objectives  were  established.  Table  4  lists  the  perfonnance 
objectives  for  the  MARS  technology,  along  with  criteria  and  metrics  for  evaluation.  Many  of  the 
performance  objectives  have  been  developed  to  parallel  those  for  the  HeliMag  system  since  an 
important  component  of  this  demonstration  is  to  compare  the  performance  of  the  MARS  to  that 
system. 

ESTCP  emplaced  test  items  at  the  Fonner  KPBR  site  to  be  used  as  a  blind  test  to  validate  the 
system’s  perfonnance.  Scoring  of  these  items  is  one  of  the  primary  metrics  for  evaluating  the 
MARS  system  performance. 

Table  4.  Performance  Objectives  for  the  MARS  Demonstration. 


Type  of 
Performance 
Objective 

Primary  Performance 
Criteria 

Expected  Performance 
(Metric) 

Actual 
Performance 
Objective  Met? 

Primary/ 

Qualitative 

Ease  of  use  and  efficiency  of 
operations  for  sensor  system 

Efficiency  and  relative  ease  of  use 

No 

Primary/ 

Quantitative 

Geo-reference  position 
accuracy 

Within  0.5  m  (determined  by  ESTCP  and 
daily  calibration  target  dipole  fit  position 
estimates) 

No 

Target  feature  definition 

Bias  and  standard  deviation  of  daily 
calibration  target  dipole  fit  parameters 
within  25  %  of  values  for  previous 

Kirtland  HeliMag  deployment 

Yes 

Primary/ 

Quantitative 

Probability  of  detection  (Pd) 
relative  to  the  HeliMag 

Pd  (MARS)  >  0.9  Pd  (HeliMag) 
Pfa(MARS)  <0.5  (based  on  comparison 
with  vehicular) 

No 

Primary/ 

Quantitative 

Detection  probability  on 
emplaced  objects 

Pd  (MARS)  >  0.9  for  items  of  105-mm 
caliber  and  greater 

No 

Secondary/ 

Quantitative 

Terrain/ 

vegetation/ infrastructure 
restrictions 

Acres  surveyed  with  MARS  >  90%  of 
planned  survey  area 

No 

Secondary/ 

Quantitative 

Operating  parameters 
(altitude,  speed,  production 
level) 

2-3  m  above  ground  level  (AGL);  68 
knots  (35  m/s);  400  acres/day 

Yes 

Secondary/ 

Quantitative 

Ease  of  use:  Navigational 
accuracy 

95  %  of  actual  flight  path  within  2  m  of 
planned  flight  path 

NA 

Secondary/ 

Quantitative 

Noise  level  (combined 
sensor/platform  sources, 
post-filtering) 

<  1  nT  on  all  sensors  determined  by  high 
altitude  test  flights 

Yes 

Secondary/ 

Quantitative 

Data  density/point  spacing 

<  0.175  m  along  track 

<  2  m  cross  track 

No 
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5.0  SITE  DESCRIPTION 


5.1  SITE  LOCATION  AND  HISTORY 

The  Former  KPBR  is  a  WWII-era  former  military  training  facility  located  about  10  miles  west  of 
Albuquerque,  New  Mexico.  This  FUDS  consists  of  a  total  of  -38,000  acres,  encompassing 
multiple  target  areas.  It  served  as  a  training  area  for  Kirtland  Air  Force  Base  during  WWII. 
WAA  demonstrations  were  conducted  on  the  5,000  acre  demonstration  site  located  on  either  side 
of  Double  Eagle  Airport.  The  physiography  and  known  munitions-use  history  of  the  study  area 
are  discussed  in  detail  in  the  Conceptual  Site  Model  (CSM)  (Versar,  2005). 

The  study  area  was  known  to  contain  three  precision  bombing  targets  identified  as  N-2,  N-3,  and 
NDIA,  as  well  as  a  SORT.  The  CSM  did  not  indicate  any  munitions-related  activity  in  the 
southern  portion  of  the  study  area.  Figure  3  shows  the  MARS  survey  area  within  the  WAA  study 
area,  along  with  infrastructure  and  other  site  features. 

5.2  SITE  GEOLOGY 

The  study  area  is  on  a  relatively  flat  terrace  at  about  6,000  feet  elevation  (mean  sea  level)  atop 
the  Rio  Puerco  Escarpment,  which  falls  away  to  the  west  of  the  site.  To  the  east  of  the  study  area, 
several  volcanic  cinder  cones  rise  about  300  ft  above  the  surrounding  terrain.  The  soils  within  the 
survey  area  are  deep,  well-drained  homogeneous  sandy  loams  formed  on  loess  parent  material 
with  low  magnetic  mineral  content. 

5.3  MUNITIONS  CONTAMINATION 

Documented  ordnance  present  on  the  site  surface  within  the  study  area  includes  the  following: 
(1)  M38A2  100-lb  practice  bombs  and  spotting  charges;  (2)  M85  100-lb  practice  bombs  and 
spotting  charges;  and  (3)  250-lb  general  purpose  HE  bombs. 

5.4  PRE-DEMONSTRATION  TESTING  AND  ANALYSIS 

Previous  testing  and  analysis  has  been  performed  at  the  Fonner  KPBR  site  under  the  existing 
ESTCP  WAA  Pilot  Program.  HeliMag  (performed  by  SKY),  LiDAR,  and  very  large  scale 
orthophotography  (performed  by  other  demonstrators)  data  were  collected  over  the  same  area 
surveyed  using  the  MARS  technology.  The  results  of  the  previous  analyses  confirmed  the 
presence  of  target  areas  identified  in  the  CSM,  and  several  additional  areas  of  concern  were 
identified  based  on  features  and  anomaly  densities  identified  in  the  WAA  analysis.  Because  both 
HeliMag  and  MARS  technologies  collect  data  using  magnetic  sensors,  comparison  of  the  two 
datasets  provides  a  cross-check  of  both  technologies. 
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Figure  3.  Former  Kirtland  Precision  Bombing  Range  with  Identified  Bombing  Targets  and 
other  Areas  of  Interest,  Including  Existing  Features  and  the  Utilities  Infrastructure. 

(Note  that  the  power  line  that  splits  the  northern  area  in  two  was  actually  buried,  so  that  it  was  possible  to  survey  that  “buffer  zone.”) 
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6.0  TEST  DESIGN 


The  MARS  system  survey  was  designed  to  demonstrate  the  system  in  the  United  States  and 
assess  the  performance  and  capabilities  relative  to  HeliMag  technology.  Because  both  HeliMag 
and  MARS  technologies  collect  data  using  magnetic  sensors,  comparison  of  the  two  datasets 
provides  a  cross-check  of  both  technologies.  Previous  testing  and  analysis  have  been 
performed  at  the  Former  KPBR  site  under  the  existing  ESTCP  WAA  Pilot  Program.  HeliMag 
(performed  by  SKY),  LiDAR,  and  very  large  scale  orthophotography  (performed  by  other 
demonstrators)  data  were  collected  over  the  same  area  surveyed  using  the  MARS  technology. 
The  results  of  the  previous  analyses  confirmed  the  presence  of  target  areas  identified  in  the 
CSM,  and  several  additional  areas  of  concern  were  identified  based  on  features  and  anomaly 
densities  identified  in  the  WAA  analysis. 

6.1  CONCEPTUAL  EXPERIMENTAL  DESIGN 

The  scope  of  the  survey  planned  at  the  Former  KPBR  included  the  three  confirmed  target 
areas  and  the  simulated  oil  refinery  target  (Figure  3).  This  area  coincides  with  the  area 
previously  surveyed  using  the  HeliMag  technology  to  provide  a  direct  comparison  of  the 
magnetic  data  collected  using  the  two  different  platforms.  A  portion  of  the  5,000  acres  flown 
by  the  HeliMag  system  was  inaccessible  because  of  infrastructure  present  at  the  site  (high 
voltage  power  lines,  etc.);  however,  data  were  collected  over  as  much  of  the  area  as  could  be 
safely  flown.  Additionally,  test  items  were  blind-seeded  at  KPBR  and  the  results  used  as  one 
of  the  primary  criteria  for  evaluating  the  MARS  system’s  detection  capabilities. 

6.2  SITE  PREPARATION 

Because  the  KPBR  site  is  a  WAA  site  and  had  been  previously  surveyed,  the  only  site 
preparation  needed  for  the  MARS  demonstration  was  the  emplacement  of  test  items  and  setup 
of  the  calibration  line.  The  field  deployment  schedule  was  coordinated  with  the  Program 
Office  schedule  for  emplacing  the  test  items  in  the  areas  of  the  site  north  and  south  of  the 
Double  Eagle  airport.  Battelle  was  conducting  a  demonstration  of  another  system  concurrent 
with  the  MARS  test,  so  setup  of  the  calibration  lines  was  coordinated  with  their  site  team.  No 
other  physical  site  preparation  was  needed  prior  to  conducting  the  survey. 

6.3  SYSTEM  SPECIFICATION 

The  MARS  system  was  deployed  on  the  CT  SW  LSA,  together  with  a  pilot  and  a  two-  to 
three-person  ground  support  team  to  operate  the  RTK  GPS  base  stations,  download  data, 
refuel  the  aircraft,  etc.  The  system  used  an  array  of  six  total-field  cesium  vapor 
magnetometers  deployed  on  a  7-m  boom  mounted  under  the  aircraft  wings.  The  planned 
aircraft  altitude  was  2-5  m,  with  an  average  forward  velocity  of  35  m  per  second  (m/s). 

6.4  DATA  COLLECTION 

The  demonstration  survey  was  conducted  between  April  24th  and  May  8th,  2007.  Daily  flight 
durations  were  planned  to  be  approximately  6  hours  per  day  (assuming  an  8-hour  daily 
window  for  flying).  Because  of  unfavorable  weather  conditions  (frequent  high  winds  and  rain 
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or  thunderstorms),  density  altitude4,  and  thermal  effects  that  were  encountered,  it  was  not 
possible  to  safely  operate  the  CT  as  many  hours  per  day  as  planned.  Daily  flight  durations 
ranged  from  1.4  hours  to  a  maximum  of  5.2  hours,  with  an  average  of  3.5  hours  for  days 
when  conditions  for  flying  were  favorable.  There  were  3  days  when  no  flights  were  conducted 
due  to  rain  and/or  high  winds.  After  approximately  a  week  into  the  field  deployment,  it  was 
apparent  that  surveying  the  planned  5,000  acres  would  not  be  feasible  within  the  projected 
schedule  and  project  budget  but  that  sufficient  data  were  being  collected  to  assess  the 
technology.  Therefore,  upon  discussion  and  with  direction  from  the  Program  Office,  the 
survey  was  truncated  after  2  weeks  of  survey  flights,  as  presented  in  the  demonstration  plan. 

During  the  survey,  data  were  collected  over  2,856  acres.  This  comprised  approximately  700 
acres  south  of  the  Double  Eagle  Airport  at  KPBR  (Figure  12  [Figures  12-16  are  included  in 
Appendix  B])  and  1,400  acres  north  of  the  airport  (Figure  13).  The  additional  650  acres  were 
split  between  the  Shooting  Range  area  in  the  western  part  of  the  site  (Figure  13),  where 
coverage  was  patchy  (400  acres),  and  re-flights  to  remedy  data  quality  problems. 

The  truncation  of  the  survey  meant  that  there  was  no  time  to  fill  in  the  gaps  in  coverage  in 
most  of  the  southern  and  northern  sections  of  the  KPBR  site.  The  result  was  that  there  were  a 
number  of  long,  thin  gaps  in  survey  coverage  (Figures  13  and  14).  Part  of  the  intended  survey 
area  in  the  southern  part  of  the  site  could  not  be  surveyed  due  to  an  FAA  exclusion  policy 
regarding  overflights  of  a  populated  building  (the  building  was  just  south  of  the  area  shown  on 
the  map  in  Figure  12). 

We  had  intended  to  survey  the  Shooting  Range  area  on  the  western  side  of  the  map  area  shown 
in  Figure  13.  However,  we  could  access  that  portion  of  the  site  only  on  certain  Mondays  and 
Tuesdays  and  before  9  A.M.  on  other  days.  From  the  map  in  Figure  13  it  is  evident  that 
certain  flights  extended  into  the  Shooting  Range  area,  while  others  had  to  terminate  just  to  the 
east  of  the  Range.  Also  evident  in  Figure  13  is  a  gap  in  the  coverage  just  north  of  the  Double 
Eagle  Airport.  This  corresponded  to  a  zone  where  we  had  to  carefully  schedule  our  flight 
activities  with  Battelle.  We  had  to  terminate  the  survey  before  we  had  the  chance  to  complete 
data  collection  over  that  part  of  the  site. 

Quality  Checks  and  Sensor  Calibration  Targets 

A  calibration  line  was  established  near  the  base  of  field  operations  just  north  of  Double  Eagle 
airport.  The  calibration  targets  were  placed  on  the  ground  surface  at  a  spacing  of 
approximately  50  m.  The  locations  of  all  calibration  items  were  surveyed  to  verify  positional 
accuracy.  A  dipole  model  was  fit  to  the  data  around  each  calibration  item,  and  the  location  and 
dipole  parameters  were  used  to  confirm  that  the  system  was  operating  correctly.  No  calibration 
targets  were  buried,  and  no  attempt  was  made  to  measure  a  probability  of  detection  from  the 
calibration  data. 

The  sensor  data  were  passed  through  a  hardware-based  60-Hz  noise  rejection  filter  before  they 
were  time-stamped.  The  filter  delay  is  fixed  and  predictable  but  we  found  that  we  needed  to 


4  This  is  the  altitude  in  the  International  Standard  Atmosphere  at  which  the  air  density  would  be  equal  to  the  actual 
air  density  at  the  place  of  observation. 
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make  small  adjustments  (on  the  order  of  0.01  to  0.02  seconds)  to  the  delay  to  obtain  the  best 
match  with  the  daily  calibration  data. 

The  difference  in  the  estimated  and  actual  positions  of  each  calibration  item  for  each 
calibration  run  is  summarized  in  Figure  4.  The  north-south  oriented  2.75-inch  rocket  had  too 
small  a  signal  to  be  reliably  inverted,  so  it  was  excluded  from  the  analysis.  The  estimated 
positions  have  a  mean  bias  of  8.6  cm  east  and  10.7  cm  north  relative  to  the  true  positions 
(Figure  4a).  Approximately  90%  of  the  time,  the  estimated  position  was  within  50  cm  of  the 
actual  position  (Figure  4b).  The  predicted  dipole  depths  vary  by  about  50-60  cm  for  each  item, 
with  no  bias  for  items  1  to  3,  a  negative  bias  for  item  4  and  positive  biases  for  items  5  and  7 
(Figure  4c).  Finally,  Figure  4d  shows  that  the  recovered  dipole  moments  are  reasonably  well 
clustered,  with  the  largest  variance  exhibited  by  the  2.75-inch  rocket.  This  shows  that  the 
system  performance  was  comparable  on  different  days. 

6.5  VALIDATION 

The  Program  Office  arranged  for  the  emplacement  of  a  number  of  rounds  in  the  southern  part 
of  the  KPBR  survey  area.  This  included  eighteen  81-mm  mortars,  thirteen  105-mm  high- 
explosive  anti-tank  (HEAT)  rounds,  seven  105-mm  projectiles  and  twenty-three  155-mm 
projectiles.  In  August,  we  submitted  a  dig-list  with  1,270  anomalies  to  the  Institute  of  Defense 
Analysis  (IDA)  and  they  calculated  detection  probabilities  based  on  a  1.5-m  detection  halo 
(Table  6).  Probabilities  of  detection  ranged  from  6%  for  the  81-mm  mortars  up  to  61%  for  the 
155-mm  projectiles.  As  mentioned  earlier,  we  experienced  a  glitch  in  the  processing  of  two 
flights  and  hence  thought  we  had  two  gaps  in  the  data  coverage  so  that  no  data  were  collected 
over  24  of  the  items.  When  these  items  are  removed  from  the  analysis,  the  results  for  the  81- 
mm  and  105-mm  rounds  do  not  change  appreciably,  whereas  the  detection  probabilities  for  the 
105-mm  and  155-mm  projectiles  increase  substantially  (to  60  and  93%,  respectively). 
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Figure  4.  Results  of  Daily  Calibrations  (with  different  days  plotted  as  different  symbols, 
and  each  calibration  item  as  a  different  color). 

(The  blue  triangles,  stars,  and  crosses  represented  calibration  results  for  the  N-S  oriented 
100-lb  bomb  on  different  days:  (a)  positional  offset  of  dipole  model  fit,  (b)  cumulative 
distribution  of  position  errors,  (c)  estimated  depth  of  calibration  items,  and  (d)  recovered 

dipole  moments.) 


Table  5.  Bias  and  Standard  Deviations  of  Daily  Calibration  Variables. 

Also  shown  for  reference  are  the  bias  and  standard  deviation  of  the  calibration  parameters  for 
the  Sky  Helimag  system  (which  was  deployed  at  the  same  site  but  with  a  different  set  of 

calibration  items) . 
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Most  manually  selected  targets  had  amplitudes  between  5  and  1 5  nT  with  very  few  target  picks 
below  5  nT.  The  detection  results  can  be  understood  by  computing  the  best  and  worst  case 
amplitudes  for  each  ordnance  item  in  Figure  5.  The  magnetic  models  for  the  ordnance  items  were 
obtained  from  the  test-stand  measurements  reported  in  Billings  et  al.  (2006).  When  the  sensor  to 
target  distance  exceeds  about  2.0  m,  the  amplitude  of  the  81-mm  mortar  is  less  than  5  nT,  even 
for  the  most  favorable  orientation.  The  5  nT  threshold  occurs  between  1.8  and  2.8  m  for  the  105 
mm  FIEAT  round  (depending  on  the  orientation).  For  the  105-mm  projectile,  the  threshold  is 
reached  at  between  2.2  and  3.8  m,  and  for  the  155-mm  projectile  between  3.2  m  to  about  4.7  m. 
These  calculations  are  made  assuming  the  item  has  no  remanent  magnetization:  this  will  act  to 
either  increase  or  decrease  the  maximum  detection  depth,  depending  on  whether  it  enhances  or 
reduces  the  total  magnetization  of  the  item. 

In  Figure  5b,  we  plot  the  average  ground-clearance  above  each  seeded  item  and  find  that  the 
sensors  were,  on  average,  between  1.4  and  2.6  m  aboveground.  For  most  items,  the  sensors  were 
between  1.6  and  1.8  m  aboveground.  Thus,  it  appears  that,  in  this  portion  of  the  site,  the  survey 
height  of  the  MARS  platform  was  comparable  to  that  expected  for  the  HeliMag  system  where  we 
would  anticipate  the  height  to  vary  from  1.5  to  2.0  m  above  the  item  (under  similar  favorable 
conditions  of  low  vegetation  and  minimal  topography). 

Table  6.  Detection  Results  for  Seeded  Ground-Truth  Items  (as  scored  by  IDA). 


Item 

Total 

In  gap 

Missed 

Detected 

Pd  (all) 

Pd 

(nogaps) 

81  mm  mortar 

18 

10 

7 

1 

6% 

13% 

105  mm  HEAT 

13 

4 

8 

1 

8% 

11% 

105  mm  Projectile 

7 

2 

2 

3 

43% 

60% 

155  mm  Projectile 

23 

8 

1 

14 

61% 

93% 

i.s 
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Figure  5.  (a)  Best  and  Worst  Case  Anomaly  Amplitudes  for  the  81-mm  Mortar,  105-mm 
HEAT  Round,  and  the  105-mm  and  155-mm  Projectiles  and  (b)  Average  Sensor  Altitude 
Above  Each  Seed  Item  (calculated  using  a  5-m  radius  centered  about  each  seed  location). 
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7.0  DATA  ANALYSIS  AND  PRODUCTS 


7.1  PREPROCESSING 

The  AG3  MK3  data  acquisition  system  time-stamps  each  of  the  sensor  readings  using  an  internal 
clock  that  is  synchronized  to  the  1  pulse-per-second  (PPS)  from  the  GPS.  A  hardware-based  60 
Hz  noise  rejection  fdter  was  used  to  suppress  power-line  noise  and  introduced  a  fixed  delay  into 
the  magnetometer  data  stream.  After  the  survey  was  completed,  the  data  were  processed  as 
follows: 

•  The  raw  data  for  a  given  survey  flight  were  time-aligned  (e.g.,  the  logged  GPS 
times  were  corrected  by  reference  to  the  PPS). 

•  Invalid  data  were  rejected  based  on  status  flags  present  in  the  raw  data  records 
(e.g.,  poor  GPS  fix)  or,  in  the  case  of  the  magnetometer  data,  a  simple  “in  range” 
test  was  used. 

•  The  GPS  geographic  position  coordinates  were  transformed  to  WGS84  Universal 
Transmercator  (UTM)  coordinates. 

•  The  GPS,  magnetometer,  and  auxiliary  sensor  data  streams  were  then  merged 
using  the  time  reference.  Using  knowledge  of  the  sensor-GPS  geometry,  each 
sensor  was  assigned  a  position  and  height  above  ellipsoid  (HAE)  based  on  the 
UTM  location  of  the  GPS  receiver,  the  roll  and  yaw  from  the  AVR,  and  the  pitch 
from  the  digital  compass.  In  practice,  we  found  that  the  pitch  measurements  were 
not  reliable  and  hence  were  discarded  from  the  processing  as  described 
previously. 

•  To  determine  the  height  above  ground,  one  of  two  methods  was  used,  depending 
on  whether  a  pre-existing  LiDAR  dataset  was  available  over  the  area: 

o  LiDAR  Digital  Elevation  Model  (DEM)  available:  The  height 
aboveground  was  obtained  by  subtracting  the  HAE  of  the  LiDAR  DEM 
from  the  HAE  of  each  sensor. 

o  No  LiDAR  DEM:  In  that  case,  the  laser  altimeter  data  were  used  to  build  a 
DEM  of  the  area.  The  DEM  was  created  by  using  the  GPS  location  and 
the  roll  and  yaw  to  determine  the  position  and  HAE  of  the  laser  reflection. 
The  height  aboveground  was  then  obtained  by  subtracting  the  HAE  of  the 
DEM  from  the  HAE  of  each  sensor. 

•  The  magnetometer  data  were  fdtered  to  remove  geological  and  temporal  trends. 

•  The  detrended  magnetometer  data  were  gridded  at  0.375  m  pixel  spacing. 

In  theory,  data  processing  rates  should  be  similar  to  HeliMag  processing,  where  data  collected 
over  300-500  acres  can  be  processed  in  less  than  4  hours  to  produce  an  initial  image.  In  practice, 
the  processing  took  a  little  longer  because  three  different  software  packages  were  used — 
SeaTerra’s  proprietary  data  processing  software,  UXOLab,  and  Geosoft  Oasis  Montaj.  Data  were 
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transferred  between  each  software  system  using  ASCII  files,  which  are  time-consuming  to  read 
and  write. 

The  initial  image  was  used  to  check  survey  coverage  and  operation  of  the  system.  If  no  problems 
were  detected  and  the  applied  filters  were  appropriate,  the  image  was  used  for  data  analysis  as 
described  in  the  section  below.  If  filters  were  deemed  inappropriate,  the  filter  parameters  were 
adjusted  and  the  images  recreated. 

7.2  TARGET  SELECTION  FOR  DETECTION 

Once  magnetic  anomaly  maps  were  created,  anomalies  were  selected.  Automatic  target  selection 
for  large-scale  surveys  such  as  this  one  has  the  advantage  of  being  objective  and  repeatable  as 
well  as  much  faster  than  manual  target  selection.  Automatic  target  selection  processes  have  been 
used  for  the  Fonner  KPBR  and  the  Fonner  Pueblo  Precision  Bombing  Ranges,  Colorado,  WAA 
Pilot  Program  demonstration  sites.  However,  automatic  target  pickers  are  not  yet  sophisticated 
enough  to  reliably  detect  closely  spaced  targets  or  targets  that  are  at  or  below  the  same  amplitude 
as  local  geologic  signal,  and  are  not  able  to  differentiate  between  our  targets  of  interest  and  local 
geologic  anomalies.  Therefore,  automatic  target  selection  routines  must  be  used  only  to  select 
targets  with  response  amplitudes  significantly  above  the  nominal  geologic  noise;  otherwise,  an 
inordinate  number  of  false  targets  are  selected.  Furthennore,  the  automatic  routines  do  not 
perform  well  in  areas  of  high  target  density.  For  this  demonstration,  we  manually  selected 
anomalies  (except  for  the  performance  comparison  with  HeliMag  and  ground-based  systems 
where  we  used  automated  detection  methods  in  order  to  be  as  objective  and  consistent  as 
possible). 

7.3  DATA  PRODUCTS 

Figures  12  and  13  show  the  processed  total-field  images  for  the  southern  and  northern  areas  with 
manually  selected  targets  overlain.  The  southern  area  targets  were  submitted  to  the  Program 
Office  in  August  2007  so  that  the  detection  performance  against  seeded  targets  could  be  accessed 
(see  Section  8).  There  were  no  target  picks  made  in  two  sections  of  the  site  corresponding  to  the 
two  flights  where  there  was  a  glitch  in  processing  (see  Section  6.5). 
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8.0  PERFORMANCE  CONFIRMATION  METHODS 


8.1  PERFORMANCE  CONFIRMATION  METHODS 

Table  7  summarizes  the  performance  confirmation  methods  used.  For  performance  confirmation 
of  metrics  related  to  probabilities  of  detection  (Pd)  and  false  alarm  (Pfa),  we  used  two  types  of 
validation  information: 

•  For  the  small  subset  of  data  where  there  was  overlapping  MARS,  vehicular,  and 
HeliMag  data,  we  used  the  vehicular  data  as  ground  truth  and  derived  Pd  and  Pfa 
assuming  the  vehicular  data  to  be  perfect  (Pd=l,  Pfa  =  0).  Note:  Since  MARS  is  a 
WAA  tool,  we  are  not  going  to  try  to  discriminate  UXO  from  clutter,  so  false 
alarms  are  defined  as  picks  where  there  are  no  metal  targets  detected  by  the 
ground  vehicular  data.  We  used  a  total-gradient  detection  algorithm  on  both  the 
HeliMag  and  MARS  data — the  threshold  was  determined  in  the  same  manner  as 
done  earlier  for  HeliMag  data  at  Pueblo.  In  this  procedure  we  plotted  the  number 
of  targets  versus  threshold — this  plot  has  an  inflection  point  where  we  start  to 
pick  into  the  noise  (i.e.,  at  lower  thresholds  the  number  of  targets  selected 
increases  radically). 

•  For  regions  with  only  MARS  and  HeliMag  data,  we  conducted  a  similar  analysis. 
Once  the  inflection  point  was  determined  for  both  systems,  we  compared  the 
following: 

o  Relative  target  density  results  (to  determine  usefulness  of  MARS  relative 
to  HeliMag) 

o  The  threshold  of  the  actual  inflection  point  (to  compare  system 
performance); 

•  For  the  ESTOP  emplaced  targets,  we  computed  the  Pd  for  a  range  of  ordnance 
types  based  on  the  emplaced  data. 

Table  8  describes  the  performance  metrics,  confirmation  criteria,  and  results. 
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Table  7.  Performance  Criteria. 


Performance  Criteria 

Description 

Type  of  Performance  Objective 

Technology  usage 

Ease  of  use  and  efficiency  of  operations.  Relative  ease  of  use  relates  to  the  ability  to 
fly  the  aircraft  at  the  prescribed  altitude,  ability  to  incorporate  the  sensors  and  DAS 
system  on  the  aircraft  platform.  Relative  efficiency  relates  to  the  ability  to 
consistently  meet  performance  objectives. 

Primary/  Qualitative  1 

Target  feature  definition 

Dipole  fit  feature  estimates  (parameters)  derived  from  data  collected  twice  daily 
over  calibration  targets.  Parameters  are  X,Y,Z,  size,  dipole  orientation. 

Primary/  Quantitative  j 

Location  accuracy 

Predicted  location  of  calibration  items  <  0.5  m  from  the  calibration  items  (including 
daily  and  ESTCP  calibration  items) 

Primary/  Quantitative 

Navigational  accuracy 

Actual  flight  path  <  2  m  from  planned  flight  path.  If  the  pilot  is  unable  to  fly  lines 
within  specification,  gaps  in  coverage  will  occur.  With  a  7-m  swatch  and  a  planned 
transect  separation  of  5  m,  an  appropriate  specification  is  2  m.  At  that  tolerance,  the 
left-  and  right-hand  sensors  on  adjacent  passes  will  overlap.  If  the  adjacent  pass  is 
also  2  m  in  error,  then  a  data  gap  will  occur. 

Secondary/  Quantitative 

Anomaly  detection/density 
distribution  fidelity 

Target  detection  probabilities/target  density  estimates  relative  to  the  HeliMag  system 

Primary/  Quantitative/  Qualitative 

MARS  survey  area  coverage 

Actual  number  of  acres  surveyed  as  a  percentage  of  planned  number  of  survey  acres: 
The  percentage  coverage  will  be  determined  by  assuming  that  a  gap  in  coverage  is 
any  area  where  the  sensors  are  >  2  m  apart  or  where  the  sensor  height  is  >  4  m  above 
ground  level. 

Secondary/  Quantitative 

Operating  parameters 
(altitude,  speed  .daily 
production  rates) 

Statistical  assessment  of  operating  parameter.  Expect  mean  values  to  be  2-3  m  AGL; 
68  knots  (35  m/s);  400  acres/day. 

Secondary/  Quantitative 

System  noise 

Accumulation  of  basal  noise  levels  from  all  contributing  sources  other  than 
environmental  sources.  These  sources  include  sensors  and  sensor  platforms, 
mechanical  motion  noise,  radio  frequencies,  etc.  calculated  by  root  mean  square. 

Secondary/  Quantitative  ! 

Data  density/point  spacing 

(Number  of  sensor  readings/sec)/  airspeed  and  cross-track  data  density;  number  of 
sensor  survey  lines/survey  area  width 

Secondary/  Quantitative 
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Table  8.  Expected  Performance  and  the  Confirmation  Methods  Employed. 


Performance  Metric 

Confirmation  Method 

Expected  Performance 

Actual  Performance 

Technology  usage 

Field  experience  using  technology  during 
demonstration  for  ease  of  use;  ability  to  meet 
performance  objectives  for  relative  efficiency. 

Relative  ease  of  use  and  efficiency 

Due  to  weather  conditions,  this  was 
not  met. 

Target  feature  definition 

Statistical  analysis  of  dipole  fit  parameters  derived 
from  data  collected  over  the  calibration  line 
targets.  These  will  be  compared  to  HeliMag  data 
collected  over  equivalent  calibration  items  at  the 
site  during  the  WAA  demonstration  survey. 

Bias  and  standard  deviation  of  dipole 
fit  parameters  for  each  target  are 
within  25%  of  HeliMag  results. 

Standard  deviations  (HeliMag  in 
parentheses) 

Depth:  0.16  m  (0.13  m) 

Size:  7  mm  (7  mm) 

Angle:  3.9°  (6.0°) 

Location  accuracy 

Statistical  analysis  of  calibration,  seed,  and 
ground-truth  items 

Better  than  0.5  m  (standard  deviation) 

0.23  m  E,  N  on  calibration 

0.48  to  0.74  m  on  seed  and  ground 
truth 

Probability  of  detection 
compared  to  HeliMag 

Comparison  with  HeliMag  data.  Pd(MARS)  will 
be  compared  to  the  Pd(HeliMag)  and  Pfa  will  be 
compared  to  vehicular  data  (see  text). 

Pd(MARS)  >  0.90  Pd(HeliMag), 

Pfa  <  0.5 

Pd(MARS)  ~0.7  to  0.8  Pd(HeliMag) 

Probability  of  detection 
on  emplaced  items 

(Number  emplaced  targets  detected)  /  (total 
number  emplaced  targets) 

Pd  >  0.9  on  105-mm  caliber  and 
greater 

Pd  =  93%  155  mm 

Pd  =  60%  105  mm 

Pd  =  13%  105  HEAT 

Navigational  accuracy 

By  comparing  actual  GPS  track  of  each  line 
against  planned  flight-path 

95%  within  2  m 

NA 

MARS  survey  area 
coverage  relative  to 
HeliMag 

The  percentage  of  acreage  surveyed  relative  to  the 
planned  coverage  will  be  calculated  in  a  GIS. 

Coverage  >  90%  planned  area 

65%  north  area 

83%  south  area 

System  noise 

The  system  noise  will  be  calculated  using  root 
mean  square  error  and  will  be  based  on  data 
collected  at  high  altitude  (>500  ft  AGL). 

<1  nT  on  all  sensors 

0.13  to  0.17  nT  outer  sensors; 

0.45  to  0.67  nT  inner  sensors 

Data  density/point 
spacing 

Calculated  from  statistical  analysis  of  survey  data. 
Will  use  mean  values  from  areas  without  large 
gaps  (>  5  m  in  width) 

0.175  m  along  track 

2  m  cross  track 

0.261  m  along  track 

Could  not  calculate  cross-track 
density 

Operating  parameters 
(altitude,  speed,  daily 
production  rate) 

Will  use  the  GPS  data  to  compute  histograms  and 
mean  values  of  altitude  and  speed.  Will  use  GIS  to 
compute  area  covered  each  day  (will  exclude 
weather  days  and  will  pro-rate  coverage  if  weather 
forces  an  early  stop  to  surveying) 

2-3  m  AGL;  68  knots  (35  m/s);  400 
acres/day 

2.27  m  average  AGL; 

72.3  knots  (37.2  m/s); 

260  acres/day  actual 

450  acres/day  prorated  for  weather 
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8.2  DETECTION  PROBABILITY  RELATIVE  TO  HELIMAG 


Expected  performance:  Pd(MARS)  >0.90  Pd(HeliMag)  at  PFa  =  0.5 
Actual  performance:  Not  met 

For  performance  assessment,  we  compared  the  MARS  (Figure  14)  and  HeliMag  data  (Figure  15) 
within  the  two  subregions  of  the  northern  survey  area  (Areas  1  and  2  on  the  maps).  Both  these 
regions  comprise  part  of  a  target  area  with  high  anomaly  density  and  also  contain  subregions  that 
had  been  previously  surveyed  with  GEO-CENTERS  (now  SAIC)  magnetometer  towed-array 
system  (Grids  la,  lb,  lc,  2a,  2b  and  3d). 

Comparison  of  MARS  and  HeliMag  density  estimates 

To  compare  the  performance  of  the  two  airborne  systems,  we  used  an  automatic  peak  detection 
routine  applied  to  total-gradient  images  derived  from  each  system.  Figure  6  plots  the  number  of 
MARS  and  HeliMag  detections  as  a  function  of  anomaly  amplitude  in  the  two  areas  used  for 
performance  comparison  (Figures  7a  and  8a  show  the  corresponding  MARS  total-gradient 
images  for  areas  1  and  2  while  Figures  7b  and  8b  show  the  corresponding  HeliMag  total-gradient 
images).  Both  systems  show  a  steady  increase  in  the  number  of  detections  as  the  amplitude  is 
decreased  to  a  certain  point,  after  which  there  is  a  rapid  rise  in  the  number  of  detections.  We  aim 
to  select  a  threshold  just  before  this  inflection  point  in  number  of  detections.  It  is  evident  that  the 
inflection  point  needs  to  be  selected  using  subjective  criteria.  However,  two  things  are  clear:  (1) 
the  HeliMag  has  a  larger  number  of  high  amplitude  targets  (particularly  for  area  2),  and  (2)  the 
inflection  point  for  the  HeliMag  is  lower  than  for  the  MARS.  For  both  areas  we  decided  to  select 
a  detection  threshold  of  3  nT/m  for  the  HeliMag.  For  the  MARS  we  selected  a  threshold  that  had 
the  same  slope  as  the  HeliMag  at  3  nT/m;  these  thresholds  corresponded  to  4.0  nT/m  for  Area  1 
and  3.8  nT/m  for  Area  2. 


Figure  6.  Number  of  MARS  and  Helimag  Anomalies  as  a 
Function  of  Amplitude  in  Areas  1  and  2. 
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To  estimate  the  distribution  of  metal  objects  across  the  study  area,  a  density  raster  was  computed 
using  a  90-m  radius  neighborhood  kernel  that  assigned  anomaly  densities  in  anomalies  per 
hectare  to  each  cell  in  the  raster.  The  MARS  and  HeliMag  density  images  in  Area  1  are 
qualitatively  similar  (Figures  7c  and  d)  in  the  southwesterly  and  southeasterly  parts  of  the  site. 
The  MARS  predicted  densities  are  generally  lower  than  the  HeliMag,  especially  in  the 
southwesterly  corner.  A  point-by-point  comparison  of  the  anomaly  densities  (Figure  9a)  reveals 
that  the  HeliMag  predictions  are  on  average  1 . 1  times  the  MARS  predictions.  After  lowering  the 
MARS  target  threshold  to  3  nT/m  (the  same  as  the  HeliMag),  the  predicted  densities  are  in  better 
agreement  (Figures  7e  and  9a),  and  the  line  of  best  fit  has  a  slope  of  around  1.  We  conclude  that 
in  Area  1  the  MARS  and  HeliMag  technologies  are  comparable.  A  site  manager  would  draw  the 
same  conclusions  about  the  metal  distribution  across  the  survey  area. 

Turning  now  to  Area  2,  we  see  that  the  MARS  and  HeliMag  density  estimates  are  significantly 
different  (Figures  8  and  9b).  Figure  6  already  revealed  that  there  were  many  fewer  detections  in 
the  MARS  data  compared  to  the  HeliMag.  Using  a  3.8  nT/m  threshold,  the  HeliMag  densities 
are,  on  average,  3.75  times  larger  than  those  predicted  by  MARS.  By  reducing  the  MARS  target 
threshold  to  2  nT/m,  the  average  discrepancy  between  predictions  is  reduced  to  1.4  times  the 
MARS  density,  but  there  is  a  large  variation  in  the  relative  densities  (presumably  due  to  the 
significant  number  of  false  alarms  at  that  threshold).  Close  inspection  of  the  spatial  distributions 
of  the  MARS  and  HeliMag  densities  reveals  considerable  discrepancies  across  the  whole  of  Area 
2.  The  differences  are  most  pronounced  in  the  target  region  to  the  far  west  of  the  area  shown. 
From  the  HeliMag  data  the  target  area  is  predicted  to  be  significantly  larger  and  with  a 
substantially  higher  concentration  of  metallic  targets. 

Systematic  differences  in  the  sensor  elevation  appear  to  be  the  main  reason  for  the  disparity  in 
the  predicted  anomaly  densities  (Figure  10).  On  average,  the  MARS  sensors  were  at  about  2.42 
m  aboveground  compared  to  1.44  m  for  the  HeliMag.  The  sensor  ground  clearance  for  both 
systems  was  significantly  higher  as  the  aircraft  flew  over  a  north-south  oriented  fence  at  about 
150  m  east.  This  presumably  caused  the  depression  in  the  HeliMag-derived  density  in  the  middle 
of  the  target  area.  The  increase  in  sensor  elevation  lasted  for  about  50  m  for  the  helicopter  and 
closer  to  100  m  for  the  CT  aircraft.  In  addition,  the  CT  aircraft  was  flying  higher  as  it  approached 
the  fence.  Between  about  130  and  180  m  north,  the  MARS  sensors  were  more  than  4  m  above 
the  ground,  and  hence  the  data  have  been  masked  (too  high  for  reliable  detection).  The  higher 
elevations  and  longer  recovery  distance  of  the  CT  aircraft  emphasizes  a  potential  weakness  of  the 
MARS  system  when  it  encounters  obstacles  or  sudden  variations  in  terrain. 

We  conclude  that  in  Area  2  the  MARS-derived  density  estimates  are  inferior  to  the  HeliMag- 
derived  estimates,  predominantly  due  to  a  higher  survey  altitude.  The  main  target  in  the  western 
part  of  the  area  is  evident  in  the  MARS  data  but  has  a  smaller  apparent  footprint  and  metal 
concentration.  In  the  lower  density  parts  of  the  site,  the  HeliMag-  and  MARS-derived  densities 
are  poorly  correlated.  We  suspect  that  areas  of  elevated  concentration  in  the  MARS  data  are  due 
predominately  to  clusters  of  false  alarms. 
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Figure  7.  Comparison  of  MARS  and  HeliMag  Densities  in  Area  1:  (a)  MARS  Total 
Gradient  Image,  (b)  HeliMag  Total  Gradient  Image,  (c)  MARS  Target  Density  with  4  nT/m 
Threshold,  (d)  HeliMag  Target  Density  with  3  nT/m  Threshold,  and  (e)  MARS  Target 

Density  with  3  nT/m  Threshold. 
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(a)  Total  gradient  from  MARS 
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(b)  Total  gradient  from  HeliMag 
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(d)  Density  from  MARS  (3.8  nT  m  threshold) 
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Figure  8.  Comparison  of  MARS  and  HeliMag  Densities  in  Area  2:  (a)  MARS  Total 
Gradient  Image,  (b)  HeliMag  Total  Gradient  Image,  (c)  MARS  Target  Density  with  2  nT/m 
Threshold,  (d)  MARS  Target  Density  with  3.8  nT/m  Threshold,  and  (e)  HeliMag  Target 

Density  with  3  nT/m  Threshold. 
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Figure  9.  Point-By-Point  Comparison  of  MARS  and  HeliMag  Anomaly  Densities  in  (a) 
Area  1  and  (b)  Area  2,  Using  Two  Different  Target  Thresholds  for  the  MARS  Data. 

(The  solid  black  line  represents  a  1-1  relationship  between  densities.) 
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Figure  10.  Comparison  of  MARS  and  HeliMag  Sensor  Elevations  in  Area  2. 
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Comparison  of  ground,  MARS  and  HeliMag 


We  turn  our  attention  now  to  the  six  subregions  in  Areas  1  and  2  that  have  coincident  ground- 
based  (GEO-CENTERS)  magnetics,  MARS,  and  HeliMag  data.  Figure  16  compares  total- 
gradient  images  for  each  system  over  one  of  the  subregions.  Also  included  are  total-gradient 
images  for  the  ground-based  system  after  upward  continuation  by  2.0  m.  These  upward 
continued  images  effectively  represent  the  expected  signal  recorded  by  an  airborne  system  flying 
at  that  elevation.  Any  item  that  causes  a  total-gradient  signal  above  a  certain  threshold  in  the 
upward  continued  data  should  be  detectable  by  an  airborne  system.  We  evaluate  the  perfonnance 
of  MARS  and  HeliMag  using  thresholds  of  2  and  4  nT/m.  These  numbers  bracket  the  inflection 
points  evident  in  the  plot  of  anomaly  threshold  versus  number  of  anomalies  (Figure  6). 

Figure  1 1  shows  receiver  operating  characteristic  (ROC)  curves  for  Areas  1  and  2  for  the  2  nT/m 
and  4  nT/m  threshold  ground-truth  data.  The  plots  show  the  probability  of  detection  against  the 
number  of  false-alarms  per  acre.  For  the  2  nT/m  ground-truth,  the  HeliMag  target  picks 
asymptote  to  Pd=0.9  at  around  five  false  alarms  per  acre,  for  both  Area  1  and  Area  2.  For  Area  1 
the  MARS  Pd=0.7  at  five  alarms  per  acre  and  rises  slowly  to  Pd=0.85  at  30  alarms  per  acre. 
MARS  detection  performance  is  significantly  worse  for  Area  2,  with  Pd=0.6  at  around  five  false 
alarms  per  acre,  rising  to  Pd=0.8  at  20  false  alarms  per  acre. 

Considering  only  those  ground-based  anomalies  with  upward  continued  amplitudes  greater  than 
4  nT/m,  improves  the  HeliMag  Pd  to  0.95  and  0.93  in  Areas  1  and  2  respectively.  These 
detection  probabilities  are  reached  very  quickly  with  false-alann  rates  of  2.5  and  1  per  acre, 
respectively.  For  Area  1,  the  MARS  Pd=0.85  at  five  false-alarms  per  acre,  rising  to  Pd=0.9  at  20 
false  alarms  per  acre.  For  Area  2,  Pd=0.75  at  five  false  alarms  per  acre,  rising  to  Pd=0.83  at  18 
false  alarms  per  acre. 

The  MARS  probabilities  of  detection  are  similar  to  those  of  the  HeliMag  data  when  upward 
continued  by  1.0  m.  The  false  alarm  rates  of  these  upward  continued  HeliMag  are  significantly 
smaller  than  those  of  MARS  data.  This  is  partly  due  to  smoothing  and  elimination  of  noise  by  the 
upward  continuation  operation. 


29 


Pnatjablty  <j  datoelon  Prob*»ltyo*  dMeeaon 


(a)  Area  1:  2  nT/m 


(b)  Area  2:  2  nTm 


(c)  Area  1 : 4  nTm  (d)  Area  2: 4  nT  m 


Figure  11.  ROC  Curves  for  the  MARS  and  HeliMag 
Using  the  Ground-Based  Data  as  Truth. 

(Results  are  split  between  Area  1  [a  and  c]  and  Area  2  [b  and  d].  Targets  were  selected  from  the 
total-gradient  of  the  ground-based  data,  upward  continued  by  2.0  m  at  thresholds  of  2  nT/m 

[a  and  b]  and  4  nT/m  [c  and  d].) 
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9.0 


COST  ASSESSMENT 


9.1  COST  MODEL 

Mobilization/demobilization,  airborne  surveys,  processing,  analysis,  and  reporting  costs  are 
summarized  in  Table  9.  As  noted,  SeaTerra’s  data  acquisition  system  and  compensation  system 
and  SKY’s  magnetometers  and  GPS  were  deployed  for  this  demonstration  survey,  and  the  daily 
rental  costs  for  that  equipment  are  reflected  in  the  reported  costs. 

Table  9.  Cost  Tracking. 


Cost  Category 

Sub  Category 

Details 

Costs  ($) 

Start-up  and 
coordination 

costs 

Technology  certification 
and  installation 

Includes  project  start-up,  sensor 
boom  engineering  and  fabrication, 
coordination  with  FAA  and  aircraft 
manufacturer  for  approval  of  aircraft 
modifications,  installation 

81,218(certification, 

installation) 

9,467(planning 

documents) 

Premobilization  testing 

Tests  conducted  December  2006  and 
April  2007 

54,589 

Site  visit  and 
coordination 

Coordination,  travel  for  site  visit  to 
KPBR  and  Pueblo  of  Isleta 

6,380 

Mobilization 

Personnel,  equipment,  and  aircraft 
mobilization  costs— Ashland,  OR,  to 
Albuquerque,  NM 

28,897 

Operating  costs 

Demonstration  survey 

Data  acquisition  and  associated  tasks, 
including  CT  operation  time,  project 
team  per  diem,  equipment  and 
subcontract  costs,  18  days  on  site 

132,769 

Data  processing 

Initial  and  secondary  processing  of 
data  (including  on-site  processing) 

63,733 

Data  analysis 

Analysis  of  MARS  data 

Demobilization 

Personnel,  equipment,  and  aircraft 
mobilization  costs— Albuquerque, 

NM,  to  Ashland,  OR 

Other  costs 

Demonstration  report 
preparation 

21 , 106(est) 

Management  and 
reporting 

Project  related  management, 
reporting,  and  contracting.  Costs 
include  In-Progress  Reviews  (IPR), 
symposium  attendance 

62,500 

Total  Demonstration  Cost 

487,896 

Acres  Characterized 

2795 

Demonstration  cost  per  acre 

174.56/acre 

Estimated  costs  for  a  production  survey 


9.2  COST  DRIVERS 

The  cost  of  an  airborne  survey  depends  on  many  factors,  including: 

•  Aircraft  costs,  which  can  vary,  depending  on  the  provider  of  the  aircraft 

•  Length  and  number  of  flight  lines  required  to  survey  the  area 
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•  Climate  and  weather  conditions,  which  can  affect  productivity 

•  Location  of  the  site,  which  can  influence  the  cost  of  logistics 

•  Amount  of  analysis  required  to  sufficiently  review  the  data. 

Aircraft  costs  are  a  major  cost  factor  for  any  airborne  survey.  Significant  variables  and  factors 
associated  with  the  mobilization,  data  acquisition,  and  demobilization  costs  include  the  cost  of 
aircraft  time  and  standby  time.  The  cost  of  aircraft  can  vary  depending  upon  the  type  of  aircraft 
and  operating  costs.  Standby  time  can  also  influence  the  cost  of  a  survey  and  is  typically 
assessed  at  the  cost  of  one  day  of  data  collection  (minimum  of  4  hours  were  used  for  these 
demonstrations),  including  aircraft  costs,  labor,  and  travel.  For  multiday  surveys,  weather  can  be 
more  of  a  variable,  and  standby  time  can  increase  costs. 

Mobilization  and  demobilization  costs  are  most  significantly  a  function  of  the  distance  from  the 
home  base  for  the  aircraft.  In  addition  to  the  cost  of  mobilizing  and  demobilizing  the  aircraft,  the 
cost  of  mobilizing  equipment  (sensors  and  GPS  equipment)  can  add  significantly  to  costs.  For 
longer  mobilization  distances,  the  costs  of  equipment  rental  for  mobilization  and  demobilization 
can  be  substantial.  Therefore,  for  a  site  requiring  a  longer  mobilization  distance,  the  mobilization 
and  demobilization  can  take  up  a  correspondingly  larger  amount  of  the  budget,  especially  for  a 
relatively  small  site. 

Data  processing  and  analysis  costs  are  generally  linear  with  project  size  and  site  complexity; 
other  influential  factors  include  the  objectives  of  the  program  and  associated  data  requirements. 
Processing  costs  and  data  deliverable  times  have  been  decreasing  with  experience  at  multiple 
sites,  automation  of  processing  and  analysis  routines,  and  increased  computing  power  resulting 
in  faster  processing. 

The  costs  reported  in  Table  9  are  for  the  demonstration  and  would  be  significantly  higher  than  a 
production  survey  with  the  MARS  system.  Table  10  presents  a  comparison  of  MARS  and 
HeliMag  costs  for  a  range  of  survey  sizes  and  mobilization  distances;  all  scenarios  assume  data 
collection  rates  of  250  acres/day.  The  costs  for  planning,  document  preparation,  management, 
and  data  processing  were  assumed  to  be  equivalent  for  the  two  technologies.  As  would  be 
expected,  the  cost  differences  occur  in  the  survey  and  mobilization  costs,  mostly  caused  by  the 
difference  in  aircraft  operations  costs,  and  to  a  lesser  extent,  equipment  costs.  Table  1 1  is  a 
summary  of  the  comparison  and  shows  that  the  MARS  costs  would  range  from  51%  to  66%  of 
the  HeliMag  costs  for  the  various  survey  sizes  and  mobilization  distances. 

Assumptions  for  the  MARS  production  survey  costs  include: 

•  Four-person  site  team,  including  CT  pilot,  geophysicist,  geophysical 
technician/ground  control  support,  and  aircraft  mechanic.  Costs  include  labor  and 
per  diem. 

•  Mobilization  and  demobilization  travel  days,  1  day  each  for  installation  and 
testing  and  deinstallation  for  the  site  team. 

•  Aircraft  operations  assume  a  rate  of  $  160/hour  with  a  minimum  4  hours/day. 
Based  on  the  KPBR  demonstration,  a  data  collection  rate  of  90  acres/hour  is  a 
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reasonable  average  for  days  without  significant  weather  issues.  Therefore,  for  the 
cost  comparison,  the  $640/day  minimum  was  assumed. 

The  volume  of  data  collected  and  processing  were  assumed  to  be  equivalent  to  that  for  HeliMag 
considering  the  standardized  data  collection  rate  and  number  of  days  onsite. 

9.3  COST  BENEFIT 

The  benefits  of  low-level  magnetometer  surveys  for  wide  area  assessment  purposes  have  been 
established  and  discussed  thoroughly  elsewhere  (e.g.,  “Demonstration  of  Helicopter  Multi- 
Towed  Array  Detection  System  (MTADS)  Magnetometry  Technology  for  the  ESTCP  Wide 
Area  Assessment  Pilot  Program”  Cost  and  Performance  Report,  Sky  Research,  2008).  This 
demonstration  project  has  shown  that  MARS  performance  is  generally  inferior  to  HeliMag  but 
that  the  costs  are  between  5 1  to  66%  of  a  HeliMag  survey.  In  the  flat,  open  areas  of  Kirtland, 
MARS  survey  altitudes  and  detection  performance  were  comparable  to  that  of  HeliMag. 
Therefore,  in  those  types  of  areas,  MARS  could  substitute  for  HeliMag  and  potentially  reduce 
survey  costs.  In  areas  with  more  topographic  complexity,  MARS  survey  altitudes  and 
performance  were  significantly  worse  than  that  of  HeliMag,  and  MARS  was  not  an  adequate 
substitute. 

The  niche  for  MARS  appears  to  be  in  flat,  open  sites  with  larger  (105  mm  or  greater)  caliber 
munitions,  particularly  when  those  sites  are  relatively  small  because  then  a  MARS  survey  is  a 
much  cheaper  alternative  than  HeliMag. 
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Table  10.  MARS-HeliMag  Cost  Comparison. 


Survey  Size:  1,000  Acres 

4  hr  each  way  to  mob/demob 

8  hr  each  way  to  mob/demob 

16  hr  each  way  to  mob/demob 

HeliMag 

MARS 

HeliMag 

MARS 

HeliMag 

MARS 

Planning,  prep,  &  project  mgt 
(including  work  plans) 

$32,000 

$32,000 

Planning,  prep,  &  project  mgt 
(including  work  plans) 

$32,000 

$32,000 

Planning,  prep,  &  project  mgt 
(including  work  plans) 

$32,000 

$32,000 

Mob/demob/install  boom 

$40,000 

$27,128 

Mob/demob/install  boom 

$72,000 

$31,351 

Mob/demob/install  boom 

$110,000 

$42,133 

Survey 

$82,000 

$35,904 

Survey 

$82,000 

$35,904 

Survey 

$82,000 

$35,904 

Data  processing  &  analysis 

$12,000 

$12,000 

Data  processing  &  analysis 

$12,000 

$12,000 

Data  processing  &  analysis 

$12,000 

$12,000 

Final  report 

$12,000 

$12,000 

Final  report 

$12,000 

$12,000 

Final  report 

$12,000 

$12,000 

Total 

$178,000 

$119,032 

Total 

$ 

210,000 

$123,255 

Total 

$248,000 

$134,037 

Cost  Per  Acre 

$178 

$119 

Cost  Per  Acre 

$210 

$123 

Cost  Per  Acre 

$248 

$134 

Survey  Size:  5,000  Acres 

4  hr  each  way  to  mob/demob 

8  hr  each  way  to  mob/demob 

16  hr  each  way  to  mob/demob 

HeliMag 

MARS 

HeliMag 

MARS 

HeliMag 

MARS 

Planning,  prep,  &  project  mgt 
(including  work  plans) 

$47,000 

$47,000 

Planning,  prep,  &  project  mgt 
(including  work  plans) 

$47,000 

$47,000 

Planning,  prep,  &  project  mgt 
(including  work  plans) 

$47,000 

$47,000 

Mob/demob/install  boom 

$40,000 

$27,128 

Mob/demob/install  boom 

$72,000 

$31,351 

Mob/demob/install  boom 

$110,000 

$42,133 

Survey 

$410,000 

$179,520 

Survey 

$410,000 

$179,520 

Survey 

$410,000 

$179,520 

Data  processing  &  analysis 

$40,000 

$40,000 

Data  processing  &  analysis 

$40,000 

$40,000 

Data  processing  &  analysis 

$40,000 

$40,000 

Final  report 

$15,000 

$15,000 

Final  report 

$15,000 

$15,000 

Final  report 

$15,000 

$15,000 

Total 

$552,000 

$308,648 

Total 

$584,000 

$312,871 

Total 

$622,000 

$323,653 

Cost  Per  Acre 

$110 

$62 

Cost  Per  Acre 

$117 

$63 

Cost  Per  Acre 

$124 

$65 

Survey  Size:  7,500  Acres 

4  hr  each  way  to  mob/demob 

8  hr  each  way  to  mob/demob 

16  hr  each  way  to  mob/demob 

HeliMag 

MARS 

HeliMag 

MARS 

HeliMag 

MARS 

Planning,  prep,  &  project  mgt 
(including  work  plans) 

$55,000 

$55,000 

Planning,  prep,  &  project  mgt 
(including  work  plans) 

$55,000 

$55,000 

Planning,  prep,  &  project  mgt 
(including  work  plans) 

$55,000 

$55,000 

Mob/demob/install  boom 

$40,000 

$27,128 

Mob/demob/install  boom 

$72,000 

$31,351 

Mob/demob/install  boom 

$110,000 

$42,133 

Survey 

$612,000 

$269,280 

Survey 

$612,000 

$269,280 

Survey 

$612,000 

$269,280 

Data  processing  &  analysis 

$54,000 

$54,000 

Data  processing  &  analysis 

$54,000 

$54,000 

Data  processing  &  analysis 

$54,000 

$54,000 

Final  report 

$20,000 

$20,000 

Final  report 

$20,000 

$20,000 

Final  report 

$20,000 

$20,000 

Total 

$781,000 

$425,408 

Total 

$813,000 

$429,631 

Total 

$851,000 

$440,413 

Cost  Per  Acre 

$104 

$57 

Cost  Per  Acre 

$108 

$57 

Cost  Per  Acre 

$113 

$59 
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Table  10.  MARS-HeliMag  Cost  Comparison  (continued). 


Survey  Size:  10,000  Acres 

4  hr  each  way  to  mob/demob 

8  hr  each  way  to  mob/demob 

16  hr  each  way  to  mob/demob 

HeliMag 

MARS 

HeliMag 

MARS 

HeliMag 

MARS 

Planning,  prep,  &  project  mgt 
(including  work  plans) 

$62,000 

$62,000 

Planning,  prep,  &  project  mgt 
(including  work  plans) 

$62,000 

$62,000 

Planning,  prep,  &  project  mgt 
(including  work  plans) 

$62,000 

$62,000 

Mob/demob/install  boom 

$40,000 

$27,128 

Mob/demob/install  boom 

$72,000 

$31,351 

Mob/demob/install  boom 

$110,000 

$42,133 

Survey 

$816,400 

$359,040 

Survey 

$816,400 

$359,040 

Survey 

$816,400 

$359,040 

Data  processing  &  analysis 

$67,000 

$67,000 

Data  processing  &  analysis 

$67,000 

$67,000 

Data  processing  &  analysis 

$67,000 

$67,000 

Final  report 

$30,000 

$30,000 

Final  report 

$30,000 

$30,000 

Final  report 

$30,000 

$30,000 

Total 

$1,015,400 

$545,168 

Total 

$1,047,400 

$549,391 

Total 

$1,085,400 

$560,173 

Cost  Per  Acre 

$102 

$55 

Cost  Per  Acre 

$105 

$55 

Cost  Per  Acre 

$109 

$56 

Table  11.  MARS-HeliMag  Cost  Comparison  Summary. 


Survey 

4  Hr  Mob/Demob 

8  Hr  Mob/Demob 

16  Hr  Mob/Demob 

Area 

HeliMag 

MARS 

Mars/HeliMag 

HeliMag 

MARS 

Mars/HeliMag 

HeliMag 

MARS 

Mars/HeliMag 

(Acres) 

($/Acre) 

($/Acre) 

Cost  (%) 

($/Acre) 

($/Acre) 

Cost  (%) 

($/Acre) 

($/Acre) 

Cost  (%) 

1,000 

178 

119 

66.9 

210 

123 

58.6 

248 

134 

54.0 

5,000 

110 

62 

56.4 

117 

63 

53.8 

124 

65 

52.4 

7,500 

104 

57 

54.8 

108 

57 

52.8 

113 

59 

52.2 

10,000 

102 

55 

53.9 

105 

55 

52.4 

109 

56 

51.3 
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Phone 

Fax 

E-Mail 
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Sky  Research,  Inc. 

112A/  2386  East  Mall 
Vancouver,  BC,  V6T  1Z3 
Canada 

Phone:  541-552-5185 

Cell:  604-506-9206 
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David  Wright 

Sky  Research,  Inc. 

9500  Kingsford  Drive 

Cary,  NC  27511 

Phone:  541-633-5255 
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Investigator 

Edgar  Schwab 

SeaTerra  GMBH 

Phone:  49-160-471-5580 

Research  Partner 

Ms.  Joy  Rogalla 

Sky  Research,  Inc. 

445  Dead  Indian  Road 
Ashland,  OR  97520 

Phone:  541-552-5104 

Project  Manager 

Mr.  Scott  Millhouse 

USACE  Engineering  and 
Support  Center  Huntsville 
Box  1600 

Huntsville,  AL  35807-4301 

Phone:  256-895-1607 

Contracting  Officer 

Technical 

Representative 
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APPENDIX  B 


SELECTED  IMAGES  OF  GROUND,  MARS,  AND  HELIMAG  DATA 


Figure  12.  KPBR  MARS  Survey  Southern  Area  Total-Field  Anomaly  Map  with  Manually  Selected  Targets  Overlain. 
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Figure  13.  KPBR  MARS  Survey  Northern  Area  Total-Field  Anomaly  Map  with  Manually  Selected  Targets  Overlain. 
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Figure  14.  Map  Showing  the  Location  of  the  Two  Areas  Selected  for  Comparison  of  the  MARS, 

HeliMag,  and  Ground-Based  Datasets. 

(There  were  six  different  areas  covered  with  the  ground-based  system  [shown  as  green  rectangles].) 
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Figure  15.  Total-Field  Data  from  a  Previous  HeliMag  Survey  of  the  Northern  Part  of  the  KPBR  Site  with  Manually  Selected 

HeliMag  Anomalies  Overlain. 
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Figure  16.  Comparison  of  (a)  Ground-Based,  (b)  Upward  Continued  Ground-Based  (2  m),  (c)  MARS, 

and  (d)  HeliMag  Data  on  Grid  la. 

(All  images  are  total-gradient  with  the  same  color-stretch.  Ground-based  targets  are  plotted  as  black-crosses;  excavation  locations  are 
plotted  as  black  circles,  and  MARS  and  HeliMag  target  picks  are  plotted  as  magenta  circles;  the  targets  along  the  linear  magnetic 

feature  and  the  western  edge  were  excluded  from  the  performance  analysis.) 
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